Active Learning in Partially Observable Markov Decision Processes
نویسندگان
چکیده
This paper examines the problem of finding an optimal policy for a Partially Observable Markov Decision Process (POMDP) when the model is not known or is only poorly specified. We propose two formulations of the problem. The first formulation relies on a model of the uncertainty that is added directly into the POMDP planning problem. This has some interesting theoretical properties, but is impractical when many of the parameters are uncertain. Our second approach, called MEDUSA, is an instance of active learning, whereby we incrementally improve the POMDP model using selected queries, while still optimizing reward. Results show a good performance of the algorithm even in large problems: the most useful parameters of the model are learned quickly and the agent still accumulates high reward throughout the process.
منابع مشابه
Active gesture recognition using partially observable Markov decision processes
We present a foveated gesture recognition system that guides an active camera to foveate salient features based on a reinforcement learning paradigm. Using vision routines previously implemented for an interactive environment, we determine the spatial location of salient body parts of a user and guide an active camera to obtain images of gestures or expressions. A hiddenstate reinforcement lear...
متن کاملConsolidated actor–critic model for partially-observable Markov decision processes
A method for consolidating the traditionally separate actor and critic neural networks in temporal difference learning for addressing partially-observable Markov decision processes (POMDPs) is presented. Simulation results for solving a five-state POMDP problem support the claim that the consolidated model achieves higher performance while reducing computational and storage requirements to appr...
متن کاملA POMDP Framework to Find Optimal Inspection and Maintenance Policies via Availability and Profit Maximization for Manufacturing Systems
Maintenance can be the factor of either increasing or decreasing system's availability, so it is valuable work to evaluate a maintenance policy from cost and availability point of view, simultaneously and according to decision maker's priorities. This study proposes a Partially Observable Markov Decision Process (POMDP) framework for a partially observable and stochastically deteriorating syste...
متن کاملA Bayesian Approach for Learning and Planning in Partially Observable Markov Decision Processes
Bayesian learning methods have recently been shown to provide an elegant solution to the exploration-exploitation trade-off in reinforcement learning. However most investigations of Bayesian reinforcement learning to date focus on the standard Markov Decision Processes (MDPs). The primary focus of this paper is to extend these ideas to the case of partially observable domains, by introducing th...
متن کاملA (Revised) Survey of Approximate Methods for Solving Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) are interesting because they provide a general framework for learning in the presence of multiple forms of uncertainty. We survey methods for learning within the POMDP framework. Because exact methods are intractable we concentrate on approximate methods. We explore two versions of the POMDP training problem: learning when a model of the P...
متن کامل